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MAXIMAL SUBBUNDLES AND GROMOV-WITTEN 

INVARIANTS 

H. LANGE AND P.E. NEWSTEAD 
To C. S. Seshadri on his 70th birthday 

Abstract. Let C be a nonsingular irreducible projective curve of genus g > 2 
defined over the complex numbers. Suppose that 1 < n < n — 1 and n'd — nd' = 
n'(n — n')(g — 1). It is known that, for the general vector bundle E of rank n and 
degree d, the maximal degree of a subbundle of E of rank n' is d! and that there 
are finitely many such subbundles. We obtain a formula for the number of these 
maximal subbundles when (n',d') = 1. For g = 2, n' = 2, we evaluate this formula 
explicitly. The numbers computed here are Gromov-Witten invariants in the sense 
of a recent paper of Ch. Okonek and A. Teleman (to appear in Commun. Math. 
Phys.) and our results answer a question raised in that paper. 



1. Introduction 

Let C be a non-singular irreducible projective curve of genus g > 2, defined over 
the complex numbers, and let E be a vector bundle over C of rank n and degree d. 
In describing the structure of E, it is important to consider the set of subbundles E' 
of E of any fixed rank n' and degree d' . In particular, for any n', there is a maximum 
value of d' for which such subbundles exist; the corresponding subbundles are called 
maximal subbundles of E. It is convenient here to write 

s(E,E') =n'd-nd'. 

We can then put 

s nl (E) = mm s(E,E'), 

the minimum being taken over all subbundles E' of rank n' . A subbundle E' is then 
a maximal subbundle if and only if s(E, E') = s n '(E). 

Note in particular that the bundle E is stable if and only if s n >(E) > for all n', 
1 < n' < n — 1. In general we call the s n >(E) degrees of stability and note that, for 
each n', we can define a stratification of the moduli space A4(n, d) of stable bundles 
of rank n and degree d by the locally closed subsets 

U <a {n,d) = {EE M(n,d)\s n ,(E) = s}. 

Note that U n > tS (n, d) can be non-empty only if s > and s = n'd mod n. 

It was proved in [|TI[] that s n /(E) < n'{n — n')g and in fact it is known Theoreme 
4.4] that the maximum value of s that occurs is given by 

s = n'{n — n')(g — 1) + e, 
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where e is the unique integer < e < n — 1 such that s = n'd mod n. The corre- 
sponding U n ' jS (n,d) is the unique open stratum of the stratification. 

When s — n'(n — n')(g — 1), a dimension counting argument shows that one can 
expect a general £ 6 M(n, <i) to have a finite number m n i(E) of maximal subbundles 
of rank n', the number being independent of E provided that it is chosen sufficiently 
generally. In the simplest case, when n! = 1, the number is known; in fact, 



mi(E) = n 

see [^|, H 03 f° r n = 2; [HI an d @, Proposition 3.9] in general). 



Our object in this paper is to obtain a formula for m n t(E) in the case (n', d') = 1. 
Indeed we use Grothendieck-Riemann-Roch and the Porteous formula to express 
Ai(n',d') as the top Chern class of a certain virtual bundle (Theorem 3.1). When 
g = 2 and n' = 2, we evaluate this Chern class and give an explicit formula for 
?7i2 (-E) (Theorem 4.1). 

Our result can be interpreted in terms of Gromov-Witten theory . Indeed, in (TJ 



the formula for m n i (E) is obtained as an application of a theorem on Gromov-Witten 
invariants, and the question of computing these invariants when n' > 1 is raised. 
Our results give an answer to this question. Our methods, however, are similar to 
those of fll5]| . Notice that these invariants were also defined in the last section of ||. 

It should also be noted that there is a mistake in flIEfl , where it is claimed that, 
when s < n'(n — n')(g — 1), the general bundle in U n ' tS (n, d) has only one maximal 
subbundle of rank n'. For s — n'(n — n')(g — 1), this is not correct, and indeed the 
proof given in |TB[ fails also for s < n'(n — n')(g — 1). However the result itself is 



true in this case; we provide a proof of this in Theorem 2.3. It should be emphasised 
that this does not invalidate any other result in [TBI 



The research for this paper was carried out during a visit by the second au- 
thor to the Mathematisches Institut der Universitat Erlangen-Niirnberg in Octo- 
ber/November 2001. He is grateful to the DFG for funding this visit and to the 
Institut for its hospitality. 

2. Uniqueness of Maximal Subbundles 



According to [|T6j, Theorem 0.1] U n i :S (n,d) is locally closed in Ai(n, d) and irre- 



ducible of dimension 

dim U n r iS (n, d) = [n 2 — n\n — n'))(g — 1) + s + 1. 
Hence it makes sense to speak of a general vector bundle in U n ^ s (n, d). 



Lemma 2.1. Let E be a general bundle in U n ' tS (n,d) with s < n'(n — n')(g — 1). 
Then E possesses only finitely many maximal subbundles E' . For any one of them 
both E' and E" = E/E' are general (as points of the moduli spaces Ai(n',d') and 
Jv[{n — n',d — d') respectively). 



Remark 2.2. Lemma 2.1 has been proved in Claim p. 495 and Lemma 1.4]. 
Similar statements have been given in [|], ^|, W. We include the proof for the conve- 
nience of the reader. 
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Proof. We show first that E' and E" are both general. If not, then either E' or E" 
(or both) depends on fewer parameters than the dimension of the corresponding 
moduli space; this is true irrespective of whether E' and E" are stable. Note also 
that h°(E"* ® E') = since E is stable. So the nontrivial extensions 

— ► E' — > E — > E" — > 
depend on a number v of parameters with 

v < dim M(n', d!) + dim M(n -n',d- d!) + h\E"* ® E') — 1 
= {g- l)(n' 2 + (n - n') 2 + n'(n - n')) + s + l 
= (g- l){n 2 - n'(n - n')) + s + 1 
= dim U n ' jS (n, d). 

This contradicts the generality of E. 

If the general E in U n > >s (n, d) possesses infinitely many maximal subbundles, then 
the same inequality holds, again a contradiction. □ 

If s < n'(n — n'){g — 1), we can be more precise: 



Theorem 2.3. A general vector bundle E e U n > tS (n,d) with s < n'(n — n')(g — 1) 
admits only one maximal subbundle of rank n' . 

Proof. For convenience we write n" = n — n', d" = d — d! . Passing if necessary to 
the dual bundle, we may assume that n' < n" . Moreover assume first, slightly more 
generally, that s < n'{n — n'){g — 1), since we want to deduce also something for 
s = n'(n — n')(g — 1). 

Fix a maximal subbundle E' e M(n', d') of E and write E" = E/E' e M(n", d"). 

So 

s = n'd" - n"d'. 

Suppose F is a second maximal subbundle of E. The composed map y? : F 
E — > is nonzero. Let G C be its (sheaf theoretic) image. Then we have the 
following commutative diagram 

-> -> F -> G ^0 

|j J. 
O^F'^F^ G ^0 

—>£"—>£'—> E 1 '" 

where the middle exact sequence is the push out of the upper exact sequence and the 
pull back of the lower sequence and the composition F — > E" is the map (p. Note 
that % and j are injective (as maps of sheaves). Conversely, given such a diagram 
with rk(F) = n' and deg(F) = d', then F is a second maximal subbundle of E of 
rank n! . Let uq and do denote the rank and degree of G, 1 < nc < n'. 

Let A nG)rfG (E") denote the set of subsheaves of £"' of rank nc and degree da, and 
define similarly A nH ^ H (E'). Note that A nG;dG (£" / ) and A nHtdlI (E') can be given a 
scheme structure by identifying them with Quot schemes. Finally denote by i* the 
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pull back map H 1 (E"* (g E') — > H 1 (G* <8> £"). According to the above remarks, it 
suffices to show that, if s < n'(n — n')(g — 1), then 

dim A na4a (E") + dim A nil4H (E') + h 1 (G* ® H) + dim Ker («*) < /i 1 (£"* ® E') (1) 



But, by Lemma 2.1 and |16, Theorem 0.2], 



dim A nGj d G (E") = n G d" - n"d G - n G (n" - n G ){g - 1) 
dimAw H (£') = n'd G -n G d'- (n 1 - n G )n G (g - 1). 

Moreover, since z* is surjective and h°(G* <S) H) = 0, F being stable, 

h\G*®H) = n 'd G -n G d' + n G (n'-n G )(g-l) 
dimi^er(z*) < n'{n" — n G )(g — 1) + s + n G d' — n'd G . 

Hence inequality (1) would follow from 

(n'n" + n G (n G -n' - n")){g - 1) + n G d" - n"d G + n'd G - n G d! + s < n'n"{g - 1) + s 

which is equivalent to 

n G (n G -ri- n")(g - 1) + n G (d" - d!) + (ri - n")d G < 0. (2) 

But Lemma 2.1 implies that n G {n' — n G )(g — 1) < s n '- na (F) < n'd G — n G d' which 
gives 

d G >^[(n'-n G )(g-l) + d'}. 
n 

Hence the left hand side of (2) is less than or equal to (here we use the fact that 

■ri < n") 

— [n"(n G - 2n') (g - 1) + d"ri - n"d'\ 

TV 

= ^ [n"(n G - n ')(g-l) + s- n'n"(g - 1)] 
u 

< — - [s — n'n"(g — 1)] (since n G < n) 

< for s < n'n"(g — 1). 

□ 

Suppose now s — n'(n — n')(g — 1). According to Lemma 2.1 there are only finitely 
many maximal subbundles for general E. We obtain as a consequence of the proof 
of Theorem 2.3 

Proposition 2.4. Let E be a general vector bundle on C with s n /(E) = n'(n — 
n')(g — 1) and n' < (n — n'). If E' and F are 2 maximal subbundles of rank n' of E, 
the composed map F — > E — > E / E 1 is injective. 

Proof. For the proof just note that in the proof of Theorem 2.3 all inequalities have 
to be equalities. In particular we must have n G = n', which implies the assertion. □ 
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3. Maximal subbundles of general bundles 

Let E be a general stable bundle of rank n and degree d on C and let n' be an 
integer, 1 < n! < n — 1, such that 

s n i{E) = n'{n — n')(g — 1) = n'd — nd'. 

We know by Lemma 2.1 that E has only a finite number of maximal subbundles. 
Our object in this section is to obtain a formula for this number when (n', d') = 1. 

We denote by M n r(E) the set of maximal subbundles of E of rank n'. This 
set can be given a scheme structure by identifying it with Grothendieck's scheme 
Quot n - n >^"(E) of quotients of E of rank n — n! and degree 

d" = d-d' = - ((n - n')d + s n .(E)) 
n 

Now let J denote the Jacobian of C consisting of line bundles of degree on C, 
and let Mq denote the moduli space of stable bundles of rank n' and with fixed 
determinant of degree d' on C. The map 

tt: JxM — >M(n',d'): (L, F) i — >L®F 

is an unramified covering of degree n' 29 . 

Let C denote a Poincare bundle on C x J and U a universal bundle on C x M 
(recall that we are assuming (n',d') — 1, so U exists). Let K denote the canonical 
bundle on C and q : C x J x M — > J x M and p : C x J x M — > C be the 
canonical projections. We denote the pullbacks of U and C to C x J x Mq by the 
same symbols and the restrictions of U and £ to {x} x J x M by U x and £ x . 

Theorem 3.1. Let E be a general bundle of rank n and degree d, and n' an integer 
with 1 < n' < n — 1. Suppose that n'd — nd' = n'{n — n'){g — 1) and that (n 1 , d') — 1. 
TTien the number of maximal subbundles of E of rank n' is given by 

m n ,{E) = ^-c top (T -£)[Jx M ] 

where S = q*(U* ®£*®p*{E® K)) and T = (& xeD (U* ® C* x ) ® C n , where D is a 
smooth canonical divisor on C . 

Proof. Note first that the conclusion is equivalent to 

#7r- 1 (M n ,(£)) = c top {T - £) [J x M ]. 

Consider the exact sequence 

— >U* ® C* ®p*E ^U* ® C* ®p*(E ® K) — > U* ® C*\ P * {D ) ® C n — > 

where D denotes any smooth canonical divisor. Note that for any maximal subbundle 
E' of E, 

deg(E'* ® E ® K) =n'd- nd' + nn'(2g - 2) > nn'(2g - 2). 
Since any such E' is stable by Lemma 2.1, E'* <g> E <S> K is semistable, so 

H\E'* ®E®K) = 0. 
Hence <8> £* ® ® X)) = and we have an exact sequence 

— ► q*(U* ® £* ® — >E — ► i? 1 ?*^* ® £* ® — > 
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with S, T as in the statement of the theorem. Note that, by Riemann-Roch, 

rk 8 = n' d — nd' + nn' (g — 1) 
= 2nn'(g - 1) - n' 2 {g - 1) 
= rk T — n' 2 (g — 1). 



Claim: 7r _1 (Af n /(S)) = {(L,F) G J x M \rk ip {L>F) < rk S - 1}. 
The claim is a consequence of the following lemma and Lemma 2.1. 

Lemma 3.2. Let E be a general bundle. Suppose n'd — nd 1 = n'{n — n')(g — 1) and 
let E' be a stable bundle of rank n' and degree d! . Then any nonzero homomorphism 
E' — > E is an injection onto a subbundle of E. Moreover h°(E'* ® E) — 1. 

Proof. If ij; : E' — ► E is injective then Imip must be saturated since it has the 
same degree as a maximal subbundle of E. Suppose that ip is not injective and let 
Ei = Imip. Since E' is stable, we have 

di d' d n — n' . d n — n\, , 

— > — = (g - 1) > (g - 1), 

n\ n n n n n 

contradicting the generality of E. Finally, if h°(E'* g) E) > 2, the morphism 

F(H°(E'* ®E)) — ► M n ,{E) 

is non-constant, since E' is stable. So M n >(E) is infinite, another contradiction. □ 

This establishes the claim. It follows by the Porteous formula (see |], p. 86]) that 
the fundamental class of 7r~ 1 (M n /(E)) in H*(J x M , Z) is given by 

[ir-\M n ,{E))\ = c top (F-£)[J x M ]. 

To complete the proof of the theorem, we must show that M n i (E) is smooth. 
Given the identification of M n >(E) with Quot n _ n > t( i"{E), it is sufficient to prove 

Lemma 3.3. Let E be a general bundle and E' any subbundle of E. Then 

H l (E'* <g> E/E') = 0. 



Proof, (due essentially to Laumon 0, see also pj). If H X {E'* ® E/E') ^ 0, then 



by Serre duality there exists a nonzero homomorphism E/E' — > E' ® K. We thus 
have a non-zero homomorphism 

E — ► E/E' — > E'®K — ► E®K. 

This homomorphism is clearly nilpotent. But, since E is general, it is very stable 
(see [ pi?| , Proposition 3.5]), i.e. it admits no nilpotent homomorphism E — > E ® K 
different from zero. This contradiction establishes the lemma and Theorem 3.1. □ 

Remark 3.4. It may be of interest to identify the precise generality conditions 
required on E. In fact we need 

(i) s nl {E) = n'{n-n'){g-l)- 

(ii) s ni (E) >n\{n — ni){g — 1) for 1 < ri\ < n' — 1; 

(iii) every maximal subbundle of E of rank n' is stable; 

(iv) M n , (E) is finite; 

(v) E is very stable. 
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The application of the Porteous formula, however, requires only the assumption that 
the relevant degeneracy locus of is finite and it then gives the length of this locus 
in its natural scheme structure. In particular, if we assume only conditions (i), (ii) 
and (iv), the formula counts (with the appropriate multiplicities) the number of 
stable maximal subbundles of E of rank n! . 



Remark 3.5. Since (n', d') = 1 and n'd — nd' = n'(n — n')(g — 1), it follows that n' 
divides n and we can then solve for d in terms of n, n' d' . Theorem 3.1 does therefore 



[14], section 3.4, Remark 4] whenever (n', d') = 1 
(r,d) = 1). In |14| there is no smoothness assumption on 
the Quot scheme, so we require only conditions (i)-(iv) of Remark 3.4; the formula 
of Theorem 3.1 then gives the length of the finite scheme Quot n __ n /^"{E). 



give an answer to the question of 
(or, in the notation of |14[ 



Remark 3.6. To apply this theorem, we note that the Chern classes of U and 
£ are well known (see for 14 and [|IJ for C). From this the Chern character of 
U* <g> C* <S> p*(E <S) K) and hence, using Grothendieck-Riemann-Roch, of £ can be 
computed. The Chern character of T is also easily computed, hence that of T — S. 
Converting this to Chern classes, we obtain Ct op (!F — £)• We now need to know only 
the intersection numbers of JxM . In principle, these can be calculated. In practice, 
it may be a difficult computation. In the final section of this paper, we will carry 
out the computation in the simplest case not previously known. 

One could of course have stated the theorem using the cohomology of M(n', d') 
rather than that of J x Mq. We have chosen the latter, because (at least in the case 
ml = 2) it is easier to work with. 

4. The case g = 2, n' = 2 

Theorem 4.1. Let E be a general bundle of rank n and degree d on a curve C of 
genus 2. Suppose that n> 4 is even and 2d + 4 = mod n with odd. Then 

m 2 (E) = -(n 2 + 2). 

Remark 4.2. Note that the right hand side of this expression is always a positive 
integer for n even. The numerical conditions are precisely what is needed to allow 
d' to be odd. When they are satisfied we can take dl to be any odd integer and then 

d='-d' + n -2. 
2 



Before starting on the proof, we recall some properties of the cohomology ring 
H*(C x J x Mo, Z). The abelian group H l (C, Z) has a natural symplectic structure. 
Let / denote the positive generator of H 2 (C, Z). We can then choose a symplectic 
basis ei, e 2 , e 3 , e 4 for i? 1 (C, Z) such that 

eie 3 = e 2 e 4 = -/ (1) 

and all the other products CiCj for i < j are 0. On C x J we can normalise £ so that 

c(£) = l + 6 (2) 
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where G H X (C, Z) ® ff^J, Z) C # 2 (C x J) can be written as 

4 

6i = / l e» ® <Pi (3) 
i=i 

with ^ G H\J y TL). We have (see U, p.335]) 

£ 2 = -20/, # 2 [J] = 2. (4) 

The variety M is given by the intersection of two quadrics in P 5 (see |12[| ). From 
this we can see that its cohomology groups are 

Z, 0, Z, Z 4 , Z, 0, z. 

If a is the positive generator of H 2 (M , Z), then a 3 [M ] = 4, so by (4) 

a 3 6 2 [J x M ] = 8. (5) 

Taking d! — 1 and normalising as in ]13| we have 

Cl (W) = a + /, C2(W) = x + 6 + a/ (6) 

with x e # 4 (M , Z) and £ 2 = ^(C, Z) ® # 3 (M , Z). In the notation of @ and 0, 
4% = a 2 — (3 and we write 

e 2 =lf, 7Gtf 6 (M ,Z). (7) 

Moreover, when g = 2, the relations of the Theorem of |7j give 

a 2 + p = 0, a 3 + 5af3 + 47 = 0. 

Hence (3 = —a 2 , 7 = a 3 , and we have from (6) 

2 

c 2 (U) = y + 6 + a/, e 2 2 = « 3 /- (8) 
We define also A G if 1 (J, Z) ® # 3 (M , Z) by 

£16 = A/- (9) 

In H*(C x J x Mq, Z), we have for dimensional reasons (noting that H 5 (M , Z) = 0) 
that the following classes are all zero: 

f 2 , a 4 , Zif, 6/, «6, aA, fl 2 A. (10) 

Finally we need to compute 6A 2 . We can write 

4 

6 = ^ei®Vi with ^eif 3 (M ,z). 
1=1 

Using this, (1) and (3), we have 

£l& = (^1^3 + ^2^4 - ^3^1 - <P4,lfa)f- 

So by (9) 

A = V?1^3 + ^2^4 - ^3^1 - ^4^2 • 

Also from (3) and (4), 

6 = -pips - V 9 2V 3 4- 
Hence 9 2 = 2(piip 3 ip2<p4:, so by (4) 

tPl<P3<P2<f4.J] = 1- 
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Now 



and 



So by (5) 



A 2 = -2(^1(^2^3 ^4 + lifiip^^i + 2y9i^4^3?/>2 
6 A 2 = -2(pi(p 3 (p 2 (p4^4^2 - 2ip 2 (p^l<P3^3^1 

(by (7)) 
(since 7 = a 3 ). 



¥ 2 i 

\d 2 ^ 



9A 2 [J x M ] = 4. 



Proof of Theorem 4-1- Tensoring by a line bundle over X if necessary, we can sup- 
pose d! — 1. By (2) and (4) we have 



= ! + & -Of 



while by (6), (8) and (10) 



ch{U) = 2 + (a + f) + (-6) + ( -^« 3 - \a 2 f 



Moreover 



So 



c/i(p*(E <g> K)) ■ p*td(C) = n + ( -n - 2 ) /. 



c/i(W* <g> £* (g) p*(£ <g> X)) • p*td(C) = 2n + [{An -4)f-na- 2<i] 



+ 
+ 
+ 



— ( —n — 2 ] a — 2nd ) f + na^i — n£ 2 



-a 2 + nA + na9) f H a" 

4 / 12 



5 1 

24" 6 



n - 7 ) « 3 / - Y2 a3 ^ 



+ 



-— a 3 0/ 
12 V 



Hence by Grothendieck-Riemann-Roch 



ch(S) = 4n - 4 + 
+ 



- I -ra-2 ) n - 2//0 



+ 



a + nA + naO 



5 1 
— n l a" 

24 6. 



+ 



a 3 6» 

12 



From (6) and (8) we obtain easily 

ch(F) 



lb n 

= 4n- 2na + -cr. 

6 
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So 



ch(T - £) = 4 + 
+ 



-n - 2 ) a + 2n# 



1 1 



6 24 



n a 



+ 



n 

12' 



n 



——a 2 — nA — na0 
4 



a 3 # 



Converting this into Chern classes, we obtain 

c to p(^ -£) = c 5 (F -£)= (±n 5 - ^n 3 ) a 3 £ 2 + n 3 0A 2 



By (5) and (11), we get 



c top (F-£)[Jx M ] = ^ 5 + ^ 3 



and Theorem 4.1 follows from Theorem 3.1. 



□ 
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